Wearable Augmented Reality Devices Research Articles

A systematic evaluation of an RTK-GPS device for wearable augmented reality

Global Positioning Satellite (GPS) systems sample points on the Earth’s surface with meter accuracy. Real-Time Kinematic (RTK) devices improve GPS performances by providing real-time correction data from ground stations, achieving centimeter accuracy. Reliable tracking approaches are essential for Augmented Reality (AR) applications, especially for outdoor scenarios, which still present unsolved challenges. AR handheld tracking capabilities have been greatly improved by integrating visual tracking approaches with RTK devices, whereas little is known about combining wearable AR interfaces with RTK systems. Although wearable AR devices are intrinsically designed for AR applications, their performance dramatically reduces in large outdoor areas, comprising the user experience. Hence, this paper provides a rigorous evaluation of a small-size RTK device that does not need any additional software integration to collect positional data. The main goal of the assessment is to verify whether its integration with a wearable AR device is advantageous or not. The evaluation has been performed considering both static and dynamic scenarios in open-sky and urban areas. The results show that the RTK device can achieve 1 cm accuracy when used in open-sky areas. In contrast, its accuracy dramatically reduces in the proximity of buildings and obstacles, showing average errors ranging from 76 to 2561%. Since wearable AR devices have an average accuracy of 2 cm, the outcomes indicate that RTK devices should be combined with wearable AR devices only when the RTK device is far from obstacles. On the contrary, the positional data should be completely avoided when barriers surround the RTK device.

ImTooth: Neural Implicit Tooth for Dental Augmented Reality.

The combination of augmented reality (AR) and medicine is an important trend in current research. The powerful display and interaction capabilities of the AR system can assist doctors to perform more complex operations. Since the tooth itself is an exposed rigid body structure, dental AR is a relatively hot research direction with application potential. However, none of the existing dental AR solutions are designed for wearable AR devices such as AR glasses. At the same time, these methods rely on high-precision scanning equipment or auxiliary positioning markers, which greatly increases the operational complexity and cost of clinical AR. In this work, we propose a simple and accurate neural-implicit model-driven dental AR system, named ImTooth, and adapted for AR glasses. Based on the modeling capabilities and differentiable optimization properties of state-of-the-art neural implicit representations, our system fuses reconstruction and registration in a single network, greatly simplifying the existing dental AR solutions and enabling reconstruction, registration, and interaction. Specifically, our method learns a scale-preserving voxel-based neural implicit model from multi-view images captured from a textureless plaster model of the tooth. Apart from color and surface, we also learn the consistent edge feature inside our representation. By leveraging the depth and edge information, our system can register the model to real images without additional training. In practice, our system uses a single Microsoft HoloLens 2 as the only sensor and display device. Experiments show that our method can reconstruct high-precision models and accomplish accurate registration. It is also robust to weak, repeating and inconsistent textures. We also show that our system can be easily integrated into dental diagnostic and therapeutic procedures, such as bracket placement guidance.

Towards Wearable Augmented Reality in Healthcare: A Comparative Survey and Analysis of Head-Mounted Displays.

Head-mounted displays (HMDs) have the potential to greatly impact the surgical field by maintaining sterile conditions in healthcare environments. Google Glass (GG) and Microsoft HoloLens (MH) are examples of optical HMDs. In this comparative survey related to wearable augmented reality (AR) technology in the medical field, we examine the current developments in wearable AR technology, as well as the medical aspects, with a specific emphasis on smart glasses and HoloLens. The authors searched recent articles (between 2017 and 2022) in the PubMed, Web of Science, Scopus, and ScienceDirect databases and a total of 37 relevant studies were considered for this analysis. The selected studies were divided into two main groups; 15 of the studies (around 41%) focused on smart glasses (e.g., Google Glass) and 22 (59%) focused on Microsoft HoloLens. Google Glass was used in various surgical specialities and preoperative settings, namely dermatology visits and nursing skill training. Moreover, Microsoft HoloLens was used in telepresence applications and holographic navigation of shoulder and gait impairment rehabilitation, among others. However, some limitations were associated with their use, such as low battery life, limited memory size, and possible ocular pain. Promising results were obtained by different studies regarding the feasibility, usability, and acceptability of using both Google Glass and Microsoft HoloLens in patient-centric settings as well as medical education and training. Further work and development of rigorous research designs are required to evaluate the efficacy and cost-effectiveness of wearable AR devices in the future.

Employee Engagement and Multimodal Behavioral Analytics, Generative Artificial Intelligence and Workplace Tracking Systems, and Wearable Augmented Reality Devices in Immersive Digital and Virtual Office Spaces

Employee Engagement and Multimodal Behavioral Analytics, Generative Artificial Intelligence and Workplace Tracking Systems, and Wearable Augmented Reality Devices in Immersive Digital and Virtual Office Spaces

Remote Collaboration and Virtual Team Performance Tools, Generative Artificial Intelligence and Adaptive Self-Organizing Systems, and Wearable Augmented Reality Devices in Immersive Work Environments

Remote Collaboration and Virtual Team Performance Tools, Generative Artificial Intelligence and Adaptive Self-Organizing Systems, and Wearable Augmented Reality Devices in Immersive Work Environments

Actuating compact wearable augmented reality devices by multifunctional artificial muscle

An artificial muscle actuator resolves practical engineering problems in compact wearable devices, which are limited to conventional actuators such as electromagnetic actuators. ing the fundamental advantages of an artificial muscle actuator provides a small-scale, high-power actuating system with a sensing capability for developing varifocal augmented reality glasses and naturally fit haptic gloves. Here, we design a shape memory alloy-based lightweight and high-power artificial muscle actuator, the so-called compliant amplified shape memory alloy actuator. Despite its light weight (0.22 g), the actuator has a high power density of 1.7 kW/kg, an actuation strain of 300% under 80 g of external payload. We show how the actuator enables image depth control and an immersive tactile response in the form of augmented reality glasses and two-way communication haptic gloves whose thin form factor and high power density can hardly be achieved by conventional actuators.

Real-World Effectiveness of Wearable Augmented Reality Device for Patients With Hearing Loss: Prospective Study

BackgroundHearing loss limits communication and social activity, and hearing aids (HAs) are an efficient rehabilitative option for improving oral communication and speech comprehension, as well as the psychosocial comfort of people with hearing loss. To overcome this problem, over-the-counter amplification devices including personal sound amplification products and wearable augmented reality devices (WARDs) have been introduced.ObjectiveThis study aimed to evaluate the clinical effectiveness of WARDs for patients with mild to moderate hearing loss.MethodsA total of 40 patients (18 men and 22 women) with mild to moderate hearing loss were enrolled prospectively in this study. All participants were instructed to wear a WARD, Galaxy Buds Pro (Samsung Electronics), at least 4 hours a day for 2 weeks, for amplifying ambient sounds. Questionnaires including the Korean version of the abbreviated profile of hearing aid benefit (K-APHAB) and the Korean adaptation of the international outcome inventory for hearing aids (K-IOI-HA) were used to assess personal satisfaction in all participants. Audiologic tests, including sound field audiometry, sound field word recognition score (WRS), and the Korean version of hearing in noise test (K-HINT), were administered to 14 of 40 patients. The tests were performed under two conditions: unaided and aided with WARDs.ResultsThe mean age of the participants was 55.4 (SD 10.7) years. After 2 weeks of the field trial, participants demonstrated a benefit of WARDs on the K-APHAB. Scores of 3 subscales of ease of communication, reverberation, and background noise were improved significantly (P<.001). However, scores regarding aversiveness were worse under the aided condition (P<.001). K-IOI-HA findings indicated high user satisfaction after the 2-week field trial. On audiologic evaluation, the K-HINT did not show significant differences between unaided and aided conditions (P=.97). However, the hearing threshold on sound field audiometry (P=.001) and the WRS (P=.002) showed significant improvements under the aided condition.ConclusionsWARDs can be beneficial for patients with mild to moderate hearing loss as a cost-effective alternative to conventional hearing aids.

Intraoperative Navigation in Plastic Surgery with Augmented Reality: A Preclinical Validation Study.

Augmented reality allows users to visualize and interact with digital images including three-dimensional holograms in the real world. This technology may have value intraoperatively by improving surgical decision-making and precision but relies on the ability to accurately align a hologram to a patient. This study aims to quantify the accuracy with which a hologram of soft tissue can be aligned to a patient and used to guide intervention. A mannequin's face was marked in a standardized fashion with 14 incision patterns in red and nine reference points in blue. A three-dimensional photograph was then taken, converted into a hologram, and uploaded to HoloLens (Verto Studio LLC, San Diego, Calif.), a wearable augmented reality device. The red markings were then erased, leaving only the blue points. The hologram was then viewed through the HoloLens in augmented reality and aligned onto the mannequin. The user then traced the overlaid red markings present on the hologram. Three-dimensional photographs of the newly marked mannequin were then taken and compared with the baseline three-dimensional photographs of the mannequin for accuracy of the red markings. This process was repeated for 15 trials (n = 15). The accuracy of the augmented reality-guided intervention, when considering all trials, was 1.35 ± 0.24 mm. Markings that were positioned laterally on the face were significantly more difficult to reproduce than those centered around the facial midline. Holographic markings can be accurately translated onto a mannequin with an average error of less than 1.4 mm. These data support the notion that augmented reality navigation may be practical and reliable for clinical integration in plastic surgery.

Towards a More Socially Sustainable Advanced Pilot Training by Integrating Wearable Augmented Reality Devices

For flying all types of turbine-engine airplanes, a pilot must undergo an intense type rating (TR) course. This study investigated the learning conditions and TR course content, and specifies the most difficult course elements that could be tackled by augmented reality (AR) applications. Because women are underrepresented in the worldwide pilot population, it is important to address gender-specific preferences and needs in the development of AR-based wearable technologies for advanced pilot training. A gender-sensitive survey of the learning conditions and course contents was conducted with 31 pilots and 22 instructors. Despite many similarities, the results confirm that there are gender-specific needs and preferences for the development of future AR-based applications for TR training. In addition, the views of both pilots’ and instructors’ are required to obtain a comprehensive assessment of the learning contents and conditions related to TR. The results also show that time pressure increased the perceived difficulty of the course for some trainees. These results are important because they indicate the directions to be taken in developing future AR-based training applications for a more learner-centered and inclusive TR training. Future directions to foster a socially sustainable development of AR-based training means for TR with special focus on gender diversity are presented.

Metaverse Employee Socialization and Operations Management, Mobile Biometric and Sentiment Data, and Wearable Augmented Reality Devices in Immersive Workspaces

Metaverse Employee Socialization and Operations Management, Mobile Biometric and Sentiment Data, and Wearable Augmented Reality Devices in Immersive Workspaces

Clinical Comparison of a Hearing Aid, a Personal Sound Amplification Product, and a Wearable Augmented Reality Device

Clinical Comparison of a Hearing Aid, a Personal Sound Amplification Product, and a Wearable Augmented Reality Device

A scalable wearable AR system for manual order picking based on warehouse floor-related navigation

In manual order picking systems, a paper list specifying the name, location, and amount of each item is often given, and this would exert mental pressure on pickers while finding target shelf bins in high-density picking environments. Augmented reality (AR) can provide a friendly alternative to improve the manual order picking performance by conveying picking information into visual instruction. Nevertheless, there is still no systematic consensus to deploy the pick-by-AR associated with actual warehouse workplaces. To establish a spatial correspondence between the visual guidance with the actual workplace, the multi-marker-based global map about the warehouse floor is established in advance. Instead of traditional single marker-based pick-by-AR methods, the warehouse floor-related map can provide an accurate and continuous navigation performance for intuitive AR guidance, allowing the picker to move freely on the warehouse floor without limiting to some certain locations. Besides, a systematic pick-by-AR solution is available by integrating the proposed method to a lightweight wearable AR device, and this easy-to-deploy pick-by-AR solution can alleviate the worker’s mental effort while doing picking action. Finally, the pick-by-AR system described in the paper is deployed in the automobile assembly line, and experimental results illustrate that the proposed method can improve the picking efficiency while reducing the picking errors compared with the current paper-based order picking.

SARA: A Microservice-Based Architecture for Cross-Platform Collaborative Augmented Reality

Augmented Reality (AR) functionalities may be effectively leveraged in collaborative service scenarios (e.g., remote maintenance, on-site building, street gaming, etc.). Standard development cycles for collaborative AR require to code for each specific visualization platform and implement the necessary control mechanisms over the shared assets. in order to face this challenge, this paper describes SARA, an architecture to support cross-platform collaborative Augmented Reality applications based on microservices. The architecture is designed to work over the concept of collaboration models which regulate the interaction and permissions of each user over the AR assets. Five of these collaboration models were initially integrated in SARA (turn, layer, ownership, hierarchy-based and unconstrained examples) and the platform enables the definition of new ones. Thanks to the reusability of its components, during the development of an application, SARA enables focusing on the application logic while avoiding the implementation of the communication protocol, data model handling and orchestration between the different, possibly heterogeneous, devices involved in the collaboration (i.e., mobile or wearable AR devices using different operating systems). to describe how to build an application based on SARA, a prototype for HoloLens and iOS devices has been implemented. the prototype is a collaborative voxel-based game in which several players work real time together on a piece of land, adding or eliminating cubes in a collaborative manner to create buildings and landscapes. Turn-based and unconstrained collaboration models are applied to regulate the interaction. the development workflow for this case study shows how the architecture serves as a framework to support the deployment of collaborative AR services, enabling the reuse of collaboration model components, agnostically handling client technologies.

A smart assistance system for cable assembly by combining wearable augmented reality with portable visual inspection

BackgroundAssembly guided by paper documents is the most widespread type used in the process of aircraft cable assembly. This process is very complicated and requires assembly workers with high-level skills. The technologies of wearable Augmented Reality (AR) and portable visual inspection can be exploited to improve the efficiency and the quality of cable assembly. MethodsIn this study, we propose a smart assistance system for cable assembly that combines wearable AR with portable visual inspection. Specifically, a portable visual device based on binocular vision and deep learning is developed to realize fast detection and recognition of cable brackets that are installed on aircraft airframes. A Convolutional Neural Network (CNN) is then developed to read the texts on cables after images are acquired from the camera of the wearable AR device. An authoring tool that was developed to create and manage the assembly process is proposed to realize visual guidance of the cable assembly process based on a wearable AR device. The system is applied to cable assembly on an aircraft bulkhead prototype. ResultsThe results show that this system can recognize the number, types, and locations of brackets, and can correctly read the text of aircraft cables. The authoring tool can assist users who lack professional programming experience in establishing a process plan, i. e., assembly outline based on AR for cable assembly. ConclusionsThe system can provide quick assembly guidance for aircraft cable with texts, images, and a 3D model. It is beneficial for reducing the dependency on paper documents, labor intensity, and the error rate.

Augmented Reality in Industry 4.0

Since the origins of Augmented Reality (AR), industry has always been one of its prominent application domains. The recent advances in both portable and wearable AR devices and the new challenges introduced by the fourth industrial revolution (renowned as industry 4.0) further enlarge the applicability of AR to improve the productiveness and to enhance the user experience. This paper provides an overview on the most important applications of AR regarding the industry domain. Key among the issues raised in this paper are the various applications of AR that enhance the user's ability to understand the movement of mobile robot, the movements of a robot arm and the forces applied by a robot. It is recommended that, in view of the rising need for both users and data privacy, technologies which compose basis for Industry 4.0 will need to change their own way of working to embrace data privacy.

Wearable Improved Vision System for Color Vision Deficiency Correction.

Color vision deficiency (CVD) is an extremely frequent vision impairment that compromises the ability to recognize colors. In order to improve color vision in a subject with CVD, we designed and developed a wearable improved vision system based on an augmented reality device. The system was validated in a clinical pilot study on 24 subjects with CVD (18 males and 6 females, aged 37.4 ± 14.2 years). The primary outcome was the improvement in the Ishihara Vision Test score with the correction proposed by our system. The Ishihara test score significantly improved (n}{}p = 0.03n) from 5.8 ± 3.0 without correction to 14.8 ± 5.0 with correction. Almost all patients showed an improvement in color vision, as shown by the increased test scores. Moreover, with our system, 12 subjects (50%) passed the vision color test as normal vision subjects. The development and preliminary validation of the proposed platform confirm that a wearable augmented-reality device could be an effective aid to improve color vision in subjects with CVD.

Privacy in an augmented reality

Google introduced Google Glass in 2012. Even though this wearable augmented reality device has only been introduced to the public in some kind of testing (beta) mode, it has managed to draw the attention of the public, partially because of the possibilities that such a device can provide and partially because of the dystopian scenarios regarding privacy. This article seeks to identify the challenges related to privacy arising from the use of this type of devices. Furthermore, it examines the existing legal framework with regards to protection of privacy in the two main markets of these devices, namely the USA and EU, and recommends some solutions that could be used as a starting point to augment the protection of privacy and at the same time give space to the new technologies to be further developed.

A heads-up display for diabetic limb salvage surgery: a view through the google looking glass.

Although the use of augmented reality has been well described over the past several years, available devices suffer from high cost, an uncomfortable form factor, suboptimal battery life, and lack an app-based developer ecosystem. This article describes the potential use of a novel, consumer-based, wearable device to assist surgeons in real time during limb preservation surgery and clinical consultation. Using routine intraoperative, clinical, and educational case examples, we describe the use of a wearable augmented reality device (Google Glass; Google, Mountain View, CA). The device facilitated hands-free, rapid communication, documentation, and consultation. An eyeglass-mounted screen form factor has the potential to improve communication, safety, and efficiency of intraoperative and clinical care. We believe this represents a natural progression toward union of medical devices with consumer technology.